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Executive Summary 


In March 2020, institutions abruptly pivoted to remote instruction, sending students, faculty, and 
staff away from college campuses in response to COVID-19. This report provides an overview of 
findings from NILOA’s national survey of assessment-related changes made in Spring 2020 in re- 
sponse to COVID-19; couples those findings with other reports released from March through July; 
and provides guidance in the form of “do’s” and “do not’s” for higher education and the field of assess- 
ment—looking beyond Fall 2020 toward what needs to be done. While reading the full report and/ 
or specific sections of interest is strongly encouraged, the executive summary provides a high-level 
overview of select findings in truncated form. 


NILOA COVID-19 Survey Finding Highlights 


In June 2020, NILOA launched a survey to capture a snapshot of assessment-related changes made 
during Spring 2020 in response to the sudden shift to remote instruction and to help determine 
remaining professional development needs. The brief questionnaire focused on changes that were 
made, potential impacts of those changes on assessment culture, and the role student voice and equi- 
ty concerns did or did not play in Spring decisions. 


Changes Made 

* 97% of respondents made changes of some kind during Spring 2020 in response to COVID-19. 

* Changes included modifying assignments and assessments, flexibility in assignment deadlines, 
shifting to pass/fail, and modifying assessment reporting deadlines. 

* Less often made changes included acceptance of alternative assignments, modifying the assess- 
ment reporting process, modifying course evaluations, shifting to credit/no credit, and changes 
to assessment roles and responsibilities. 

* The majority of institutions made between 3 to4 changes. 


Concerns Raised 
* 75% of respondents felt the changes would not negatively impact the assessment culture of 
their institution. 
° The 25% with concerns worried about increased work demands, shifting assessment further 
away from teaching and learning, and accuracy of measures of learning. 


How Decisions Were Made 

* Respondents agreed that assessment-related changes were undertaken to address student needs. 

* Concerns about students’ differential access to technology, alongside concerns about students’ 
ability to learn in remote environments were determining factors in making decisions about 
what to change. 

* Respondents also agreed, however less strongly, that equity concerns drove decisions. However, 
information gathered from students was less likely to influence decisions on what to change, 
and students were less likely to be asked to identify their needs prior to decisions being made. 


Changes that Should Continue to Support Learning 
Survey respondents indicated five areas of change that should continue beyond Spring 2020: 


1. Increased Flexibility 
2. Empathy and Use of Student Voice 
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3. Alternative Measures 
4. Addressing Inequities 
5. Planning for the Future 


Remaining Professional Development Needs 
42% of respondents stated that all of their professional development needs had been met either in 
the Spring or over the summer. Remaining needs included: 


¢ Alternative Assessment Examples 

* Best Practices for Assessing Online 

* Student Affairs Assessment in Remote Settings 
* Qualitative Assessment Support and Examples 


Part IT: Intersections with other Survey Reports and Findings 


Other survey reports found similarities in changes made, along with increased use of student feed- 
back in an effort to look internally as opposed to benchmarking externally, when making decisions on 
how best to support students, faculty, and staff. Concerns that existed pre-COVID were amplified, 
basic student needs were not met, and the rates at which they were not met were nearly double for 
students of color; raising concerns on students’ ability to engage in learning based on Maslow’s (1943) 
hierarchy of needs. What is most concerning, is that in the NILOA survey for Spring 2020, concerns 
about equity, including student voices, and listening to those voices before making changes was some- 
thing with which less than 30% of respondents strongly agreed. Yet, 90% of university presidents 
reported being very concerned about the inequitable impact on minority students, while only 45% 
felt they were successful in ensuring equitable access (Inside Higher Ed, 2020). 


Part III: Do’s and Don'ts for the Future 


1. Do not forget that we are in a pandemic. Still. Do not forget that it is also an inequitable pan- 
demic. 

2. Do not cause further harm. Do not support, enable, or endorse policies that perpetuate further 
inequities or fuel negative perceptions of students. 

3. Do not ask students for their approval of a decision that has already been made. Instead, en- 
gage with them in advance to help determine a solution. 

4. Do not require a higher-level of proof of learning in an online class than you would normally 
require in a face-to-face setting. 

5. Do not forget that this is not the educational experience students wanted or expected. Nor 
is this a test of online education. And in case you were wondering, it still will not be “online 
education” in the Fall. It will continue to be a derivative of emergency remote teaching and 
learning. 

6. Do use learning outcomes as a guide and means to design and focus educational offerings. 

. Do listen to student voices AND respond accordingly. 

. Do modify assignments and assessments in ways that are flexible, utilize low-bandwidth, and 

are based in the principles of equitable assessment. 
9. Do be aware of and address systemic inequities. 
10. Do engage in trauma-informed and healing-centered pedagogy and assessment. 
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Assessment During a Crisis: 
Responding to a Global Pandemic 


Natasha A. Jankowski 


In March 2020, institutions abruptly pivoted to remote instruction, 
sending students, faculty, and staff away from college campuses in 
response to COVID-19. Immediately, various organizations created 
resources, webinars, and administered surveys to capture, document, 
and inform the massive changes unfolding in real time. Free resources 
were plentiful and virtual, creating a sea of overwhelming options. At 
the National Institute for Learning Outcomes Assessment (NILOA), 
we joined in, holding weekly webinars including two community check- 
ins where we synthesized what we were hearing and learning from 97% 
the shifting ground below our professional feet. We released targeted 
information and tools focused on supporting conversations for Fall 
2020 planning including virtual assignment design conversations, 
transparency of learning outcomes, telling the story of the decisions 
that were made with support of communication toolkits, and using 
curriculum mapping and alignment in future program planning made changes 
(NILOA, 20202; 2020b; 2020c). We curated a Google doc of resources 

compiled from various groups and organizations, regularly updating 

materials as conversations changed and new issues emerged. I joined 

in the experience of pivoting to remote instruction, moving my Spring 

2020 graduate class to a virtual situation while students were away for 

Spring Break, not knowing at the time that the class prior had been 

our last class together in person. 


To support the pivot, sharing of resources was rampant, and while the 
possibility of a Fall that looked somewhat familiar was on the horizon 
a resurgence of the virus was close behind. Nipping at the heels of the 
virus spread came protests against police brutality, systemic racism, 
and racial inequality raising the banner that Black Lives Matter; as 
well as the looming specter of a national election in November. We 
find ourselves now entering August, the virus is surging, and Fall 2020 
is questionable in terms of what it will look like and who will come. 


Due to where we now find ourselves, this report is designed to 
accomplish three separate goals. The first, is to provide findings from 
NILOAs national survey of assessment-related changes made in Spring 
2020. The second, is to couple those findings with the multitude 
of other reports and survey findings on COVID-19 released from 
March through July to provide a synthesis of the current landscape of 
information. The last is to provide guidance in the form of “do’s” and 
“do not’s” for higher education and the field of assessment moving 


forward—looking beyond Fall 2020 toward what needs to be done. 
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NILOA COVID-19 Survey Findings 


In June 2020, NILOA launched a survey designed to capture a 
snapshot of assessment-related changes made during Spring 2020 
in response to the sudden shift to remote instruction and to help 
determine remaining professional development needs. The brief 
questionnaire focused on changes that were made, potential impacts 
of those changes on assessment culture, and the role student voice 
and equity concerns did or did not play in Spring decisions. A total 
of 834 responses were received, which led to a cleaned data set of 
813 responses. For information on the respondents and analyses see 
Appendix A, for a copy of the questionnaire see Appendix B. 


Of the respondents, 97% made changes of some kind during Spring 
2020 in response to COVID-19. That number is in alignment with 
other surveys reporting 96% of respondents making some type of 
adjustment to their assessment activities in response to the pandemic 
(Rice, 2020). The remainder of the findings in this section focus upon 
the 787 responses—the 97%—who made changes of some kind. 


While 97% indicated that changes were made, what sort of changes 
occurred? The most often mentioned change was modifications to 
assignments or assessments and flexibility in assignment deadlines (Figure 1). 
All the changes listed in Figure 2 are in alignment with those identified 
in other survey reports on COVID-19 (Blankstein, Frederick, & Wolff 
Eisenberg, 2020; Means et al, 2020; Rice, 2020, Watermark, 2020). 


66% 


of respondents indicated 
more than one assessment- 
related change in response 
to COVID-19. 


ModifyAssignments or Assessments i 509, 
Flexibility in Assignment Deadlines (EE 49% 


Shift to pass/fail for Spring ES 30% 
Modify Assessment Report Deadlines 35% 


Accept Alternative Assignments 3 32% 
Modify Assessment Reporting Process/Questions (I 29% 
Modify Course Evaluations 9 18% 


Shift to credit/no credit for Spring [J 11% 


Changes to Assessment Roles/Responsibilities Jammy 11% 


0% 10% 20% 30% 40% % CO% 


Figure 1. Changes made in Spring 2020 in response to COVID-19. 
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However, 116 respondents indicated that “other” changes were made. 
In analyzing the responses in the “other” category, the comments 
provided additional clarification on respondent selections. For 
instance, course evaluations would not be used for personnel decisions, 
or administrators would only see the faculty course evaluations if 
faculty gave approval—an approach in alignment with an American 
Sociological Statement (2020) on the faculty review and reappointment 
process during COVID-19, endorsed by over 50 different disciplinary 
associations. Respondents shared requirements about pass/fail 
options, explaining that it was not a mandatory shift automatically 
applied for all students, but an option provided to students with a 
decision date much later in the semester, in some cases, made after 
students saw their final grade. Others shared additional questions 
that were asked of graduating seniors, explained how they suspended 
program reviews for the semester, listed the additional surveys created 
to examine student and faculty experiences, raised concerns about 
labs and clinical assessments, and expressed disappointment that 
faculty group scoring of student work would now occur virtually. Only 
3 respondents indicated that assessment had simply been cancelled. 


Of the 787 institutions that reported changes were made, 66% 
identified more than one assessment-related change in response to 
COVID-19 with the majority making between 3-4 changes. However, 
and surprisingly, there was no consistency in the combinations 
of changes made, no groupings or clustering of responses, and no 
relationship between the types of changes made and answers on 
perceptions of those changes and/or how those changes were made. 
There were differences by variables of interest such as selectivity, 
Minority-Serving Institution status, urbanicity, residential campus, 
and institutional type. 


Differences in Assessment- Related Changes 


Due to the inequitable impacts of COVID-19 on race, ethnicity, 
and socio-economic status, survey responses were examined on 
institutional selectivity and Minority-Serving Institution variables 
to explore whether constructs of prestige, perceptions of quality, 
and systemic racial inequities limited institutional change options. 
As indicated in Table 1, inclusive institutions were more likely than 
selective institutions to provide flexibility in assignment deadlines, 
while selective institutions were more likely than inclusive institutions 
to shift to pass/fail or credit/no credit, modify assignments, accept 
alternative assignments, and modify assessment reporting processes. * 


1. Regarding public, private and for-profit institutional differences, the initial data run indicated numerous 
statistical differences between for-profit responses and those of public and private institutions. However, 
there were only n = 11 forprofits in the data set, suggesting caution in sharing of the differences. Thus, 
the only item shared is where there was a statistically significant difference between public and private 
institutions. 


Modified 
Course 
Evaluations 


20% of 
Public 
Institutions 


27% of 
For-Profit 
Institutions 


Public institutions were the 
least likely to modify course 
evaluations. 
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Changes Made 
Modify Assignments or Assessments 
Flexibility in Assignment Deadlines 


Accept Alternative Assignments 
Modify Assessment Reporting Process or Questions 
Shift to Credit/No Credit 


Table 1. Statistically significant changes made in Spring 2020 in response to COVID-19 by institutional 
selectivity. 


Shift to Pass/Fail 


Table 2 outlines statistically significant differences between 
changes made by Minority-Serving Institutions (MSIs) compared 
to Predominantly White Institutions (PWIs). PWIs were more 
likely than MSIs to modify assignments or assessments, modify 
the assessment reporting process, and modify course evaluations. 
However, MSIs were more likely to shift to credit/no credit options for 
students than PW/Is. On all other identified changes, no statistically 
significant differences were found. 


Institutions White Institutions 
Modify Course Evaluations 


Table 2. Statistically significant changes made in Spring 2020 in response to COVID-19 by Minority- 
Serving Institution or Predominantly White Institution. 


Given concerns about connectivity issues, bandwidth, and Internet 
access for rural students, responses were examined by urban and rural 
institutions. Rural institutions were more likely than urban to modify 
course evaluations, but less likely than urban institutions to shift to 
pass/fail and modify assignments or assessments (Table 3). 


Modify Course Evaluations 


Table 3. Statistically significant changes made in Spring 2020 in response to COVID-19 by Rural or Urban 
location. 


Due to possible differences in Spring assessment-related changes 
for residential campuses that had to send students home, and non- 
residential campuses that did not, responses were examined by 
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residential status (Table 4). Residential campuses were more likely 
than nonrresidential campuses to provide flexibility in assignment 
deadlines, shift to pass/fail, accept alternative assignments, and to 
modify assessment reporting processes and course evaluations. 


Changes Made Non-Residential 
Flexibility in Assignment Deadlines 


Shift to Pass/Fail 41% 


Accept Alternative Assignments 
Modify Assessment Reporting Process or Questions 


Modify Course Evaluations 20% 


Table 4. Statistically significant changes made in Spring 2020 in response to COVID-19 by Residential and 
Non-Residential status. 


In addition to the variables explored thus far, differences have been 
consistently seen by institutional type in prior NILOA surveys 
(Jankowski, Timmer, Kinzie, & Kuh, 2018; Kuh, Jankowski, Ikenberry, 
& Kinzie, 2014; Kuh & Ikenberry, 2009). When examining changes 
made by Associate, Baccalaureate, Master, Doctoral, and Special 
institutions, statistically significant differences were found for each 
choice item except for modification of assessment reporting processes 


(Figure 2). 


¢ Associate degree granting institutions (33%) were /east likely 
to accept alternative assignments, while Master’s institutions 
(50%) were the most likely, however, Associate degree granting 
institutions (74%) were more likely than all other types to be 
flexible in submission deadlines for assignments. 

* Doctoral institutions (75%) and Specialized institutions 
(85%) were more likely than other institutional type to modify 
assignments or assessments, while Specialized institutions were 
most likely to report changes to assessment-related roles and 
responsibilities (24%). 

¢ Master’s degree granting institutions (21%) were more likely to shift 
to credit/no credit than Associate degree granting institutions 
(10%), while Doctoral degree granting institutions (57%) were 
more likely than Baccalaureate (45%) and Associate (38%) to shift 
to pass/fail for the Spring semester/term. 

¢ Baccalaureate degree granting institutions (54%) and Associate 
degree granting institutions (51%) were more likely than all other 
types to make changes to the timing and submission of assessment 
reports, while Baccalaureate degree granting institutions (3%) 
were more likely than Associate (13%) to modify course evaluation 
questions. 
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Changes Made in Spring 2020 in Response to COVID-19 


*Modify Assignments or Assessments 


*Flexiblity in Assignment Deadlines 


*Accept Alternative Assignments 


Modify Assessment Report Deadlines 


*Modify Course Evaluations 


Modify Assessment Reporting Process / Questions 


*Shift to pass/fail for Spring 


*Changes to Assessment Roles/Responsibilities 


*Shift to credit/no credit for Spring 


0% 10% 20% 30% 40% 50% 60% 70% 80% 9% 


mSpecial mDoctoral mMaster Baccalaureate m Associate 


Figure 2. Changes made in Spring 2020 in response to COVID-19 by Institutional Type, * indicates a 
statically significant difference. 
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Concerns about Impact of Changes 


While various assessment-related changes were made in response 
to COVID-19 during Spring 2020, the questionnaire inquired as to 
whether respondents were concerned about any possible negative 
impacts of those changes on the assessment culture of the institution. 
Our concern was whether certain decisions would be perceived as 
or serve to set back assessment culture development or reinforce 
perceptions of assessment as a compliance activity. Thankfully, 75% of 
respondents were NOT concerned that the changes made in response 
to COVID-19 would negatively impact the culture of assessment at 
the institution, while 25% were concerned. 


¢ Associate degree granting institutions (29%) were more concerned 
about potential negative impacts to assessment culture than all 
other institutional types. 

¢ Faculty were more concerned (29%) about negative impacts than 
administrators (21%) and assessment professionals (21%). 

* Nonrresidential campuses (29%) were more concerned than 
residential campuses. 

* Public institutions (26%) were more concerned than private (15%) 
and for-profit institutions (18%). 

* No differences were found by institutional selectivity, MSI/PW1, 
or rural/urban. 


For respondents who indicated concern, an open-ended follow-up 
question inquired as to what the concern might entail. Of the 137 
respondents indicating concern, 120 provided responses. Concerns 
focused on three main areas: ability to meet increased work demands; 
separation of assessment from teaching and learning; and issues of 
measurement and data accuracy. 


Increased work demands. Respondents reported feeling 
overworked, mentioning that prior to COVID-19 they were already 
operating beyond capacity and that current demands and requirements 
pushed them over the limit. Concerns were raised about mental health, 
ability to stay focused and meet work-related requests and needs, 
and that without constant attention the work of assessment would 
be stalled. In-person meetings with faculty and administrators were 
cancelled, shifting attention away from group dialogue on assessment 
data and onto assessment professionals to review data and provide 
direction to departments, colleges, and the institution. Respondents 
expressed concerns that upon return to campus, assessment would 
remain the purview of the office of assessment and not something 
done collectively throughout an institution, stalling and setting back 
cultural development on the role and purposes of assessment over 
time. 


f 
I 
\ 


ou t= 


» 
29% \ 


of faculty were concerned 
about negative impacts of i 
assessment-related changes 

on assessment culture. d 


29% 

of associate degree granting 
institutions were concerned 
about negative impacts on 
assessment culture. 


& 


~ 
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Assessment divorced from teaching and learning. Respondents 
shared that in the abrupt emergency pivot to remote instruction, 
assessment was pushed to the side, sending signals to faculty that 
assessment was not important or helpful in the shift to remote 
instruction, serving to distance assessment from teaching and learning. 
Professional development offerings focused on online instruction and 
technology, not on assessment or learning. As one respondent wrote, 
“without a focus on assessment, faculty are improving instruction 
based on what?” Others raised concerns that administrators pointed 
to assessment as something to which “overworked faculty have 
earned a pass on this year”, fostering feelings of uncertainty on where 
assessment fit within the institution or how it might restart when it 
is “now seen as optional by faculty.” As one respondent wrote, “It is a 


“It is a delicate 


delicate dance of flexibility, compassion, normalcy, and compliance.” dance of 

Accuracy of measures of learning. With students’ attention drawn flexibility, 

in various directions, respondents raised concerns about data being ° 
compromised, students cheating or falsifying work, and having compassion, 
accurate representations of learning from Spring 2020. Others raised normalcy, and 
concerns about issues of equity in demonstrating learning and faculty { ‘is 
inflexibility to responsive and different measures. One respondent compliance. 
wrote, “before students even logged in remotely the first time, faculty 

believed academic standards were lowered, and that instead of making ~NILOA Survey Respondent 


changes to how we assess students, they reverted to more historical 
means of testing, proctoring, and reducing perceptions of cheating as 
opposed to changing assessment to reflect reality.” Others lamented 
that the only time they were contacted by faculty was for information 
on preventing cheating and/or information on how to implement auto- 
graded multiple choice/completion assessments, another respondent 
stated, “rather than summative projects or practicums that better 
gauge skills-based learning—it is like faculty forgot everything they 
knew about meaningful assessment.” 


Throughout the open-ended responses, a distinction was raised 
between the annual assessment reporting process at an institution 
versus the ongoing assessment of student learning in courses and 
programs. While assessment reporting processes were stalled or 
delayed, the assessment of student learning should be ongoing and was 
seen by respondents as instrumental to Fall planning; an important 
distinction between the reporting process and the improvement cycle 
that may have been missed by faculty and administrators in the pivot 
to remote instruction. 
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How Decisions Were Made 


NILOA has been interested in exploring the relationship between 
equity and assessment (Montenegro & Jankowski, 2017) as well as 
placing the student in a more central role in assessment activities 
(Montenegro & Jankowski, 2020). For these reasons, a series of 
statements regarding possible drivers of assessment-related changes 
focused on students and equity were posed to survey takers (Figure 
3). This is not to say these are the only drivers of decisions or groups 
with which to confer. Faculty and staff, and even health departments 
or state mandates/guidance were drivers of changes made in Spring 
2020. Instead, the statements serve to draw attention to the need to 
consider equity and students in light of evidence of worsening equity 
gaps and magnifying of inequities for students (Gallagher, 2020; 
Mangan, 2020). 


Assessment Related Changes were Undertaken 


to Address Student Needs ee OH 


Concerns About Students’ Differential Access 


ta A340, 
to Technology were Determining Factors fl 


Concerns About Students’ Ability to Learn in 


: shes 38% L : 
Remote Environments Drove Decisions a 


Equity Concerns Drove Decisions = 


Students were Invited to Identify Needs 


70, 
Prior to Decisions Being Made | 


Information Gathered from Students 


Influenced Decisions on What toChange | 


0% 20% 40% 60% 80% 100% 


mStronglyAgree @Agree mDisagree StronglyDisagree gN/A 


Figure 3. Responses for those who indicated that changes were made regarding strength of agreement with 
the student and equity statements. 


Of concern, 8% marked “Not Applicable” that equity concerns drove 
decisions to make changes and 17% disagreed to varying degrees— 
meaning that a quarter of respondents did not agree that decisions 
made considered equity. Further, 37% disagreed and 13% marked 
“Not Applicable” that students were invited to identify needs prior 
to decisions being made. This may have been due to timing and some 
decisions—such as moving to remote—that would have been made 
without student involvement. Even among those that gathered infor 
mation from students, 27% disagreed that the information influenced 
what to change. 


The Data Dive Box on the following page provides a brief overview 
of various statistical differences in responses by variables of interest. 
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Data DiveBox 


Addressing Students’ Needs 

Private institutions (50%) were more likely than public institutions (42%) to strongly agree, and 
Baccalaureate institutions (94%) were the more likely than all other institutional types to agree that 
changes were undertaken to address students’ needs. No statistically significant results were found by 
position of respondent (administrator, assessment professional, or faculty), institutional selectivity, 
urbanicity, residential status, or classification as an MSI regarding assessment-related changes being 
undertaken to address students’ needs. 


Equity Driving Decisions 

Faculty 6%) were more likely than administrators (26%) and assessment professionals (26%) to 
indicate that they strongly agreed that equity concerns drove decisions, while assessment professionals 
(18%) were more likely than administrators (41%) and faculty 0%) to disagree that that was the 
case. No differences were found by selectivity, residential status, or classification as an MSI. Private 
institutions (52%) were more likely than public institutions (44%) to agree, while urban institutions 
(30%) were more likely to strongly agree than rural institutions (22%) that equity concerns drove 
decisions. Lastly, Baccalaureate institutions (80%) were more likely than all other institutional types 
to agree that equity concerns drove decisions to make changes. 


Differential Technology Access 

Inclusive institutions (47%) were more likely than selective institutions 9%) to strongly agree that 
concerns for students’ differential access to technology (including reliable Internet) were determining 
factors in making changes. Assessment professionals (16%) were more likely than administrators and 
faculty to disagree that was the case, while Master’s institutions (52%) were the most likely to agree, 
and Doctoral institutions (5%) the least likely. No differences were found by MSI, control, urban/ 
rural, or residential. 


Ability to Learn in Remote Environment 

Faculty (43%) were more likely than all other positions to strongly agree that concerns about students’ 
ability to learn in their remote environments drove decisions to make changes. Inclusive institutions 
(43%) more likely than selective 35%) to strongly agree, and Baccalaureate (47%) were more likely to 
strongly agree than Doctoral institutions 1%). No differences were found by control, urban/rural, 
MSI, or residential status. 


Identify Needs Prior 

39% of faculty disagreed that students were invited to identify needs via survey, focus groups, phone 
calls, etc. prior to decisions being made. PWIs (40%) were more likely to disagree than MSIs (23%) 
that students were invited to identify needs prior, and Doctoral institutions were the most likely to 
indicate “Not Applicable” (6%) as a response. No differences were found by selectivity, control, urban/ 
rural, or residential status. 


Student Data Influenced Decisions 

Minority-Serving Institutions (70%) were more likely than PWIs (57%) to not only agree, but strongly 
agree that information gathered from students influenced decisions on what to change. Baccalaureate 
(27%) and Master’s institutions (27%) were more likely than doctoral institutions (17%) to agree. No 
differences were found by selectivity, residential status, urban/rural, position type, or control. 
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The good news is, 90% agreed that assessment related changes 
were undertaken to address students needs. Further, 85% agreed 
that concerns about students’ differential access to technology were 
drivers of changes made, along with 85% agreeing that concerns about 
students’ ability to learn in remote environments was a driver. Yet, 
only 51% agreed that students were invited to identify their needs 
prior to decisions being made, and only 58% agreed that information 
gathered from students actually influenced what to change. 


Changes that Should Continue to Support Learning 


While various assessment-related changes were made in Spring 2020 
in response to COVID-19, not all of the changes were temporary or 
should be. To determine which changes respondents wanted to see 
continue beyond Spring 2020, we posed an open-ended question and 
358 respondents offered replies that fell into five different, but inter 
related, categories: 


1. Increased Flexibility 

2. Empathy and Use of Student Voice 
3. Alternative Measures 

4. Addressing Inequities 

5. Planning for the Future 


Increased Flexibility 


Increased flexibility included comments that focused upon providing 
additional options to students that reflect life’s realities and broadened 
where faculty and the institution thought students were able to learn. 
They included additional options for student choice, with a focus 
upon course designs allowing flexibility of assessments per students’ 
choices. Increased flexibility was also directed towards faculty who 
were able to try different methods when teaching or assessing student 
learning. It also included a focus on offering multiple learning options 
such as dual synchronized options to attend class in multiple ways 
because “some problems are beyond student control and we shouldn’t 
cause harm because of that.” Increased flexibility was also linked to a 
realization that the “whole mentality of you have to show up to learn 
in front of me so I can see you are paying attention means that your 
health is not as important as my class, and I just can’t support that 
argument anymore.” 
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66 


One of our faculty members offered her students the opportunity to retake their exams 
online because of the shift to remote learning. Unsurprisingly, the students who retook 
their exams scored higher after reading the feedback from the first exam and having 
the opportunity to take it again. Not only did the students earn significantly higher 
grades, but the instructors course evaluation feedback also reflected that the students 
appreciated the support. When she compared both students’ achievement and her past 
scores on course evaluations, both had improved from previous times she taught the 
course. If the goal of higher education is truly for students to learn, I would hope that 
our faculty who discovered new strategies to assist students achieve their goals would 
continue to offer them in the future. I also think that offering students more time 
to decide if they want to change their grade from a letter grade to pass/fail should be 
continued. After this experience, the deadlines seem pretty arbitrary; focused more 
on what’s easiest to manage—in terms of institutional red tape—than what is most 


supportive of students and their needs. 
-NILOA Survey Respondent 


99 


Empathy and Use of Student Voice 


Second only to comments on being flexible were statements supportive 
of increased use of student voice and a stronger commitment to 
empathy with students. Using student voices to make decisions, 
engaging in formative feedback and after-action review with students 
about what is working for them and what is not, and creatively solving 
problems in partnership with students were key elements of these 
comments. Additionally, shifting the mindset to engage with students 
to determine what is feasible and doable, as well as a desire to keep 
policies that were student-focused and supportive of student health, 
well-being, learning, success, finances, and engagement with the 
university community were commonalities. A focus upon compassion 
and concern for students showed itself, as one respondent wrote, “as 
though a bond grew between students and faculty as a result of the 
shared experience of actually having to learn together—I don’t want to 
see that go away.” That bond appeared in end of term survey responses, 
where “It was like our faculty suddenly remembered that they actually 
care for students, so much so that the most common answer students 
reported in a survey at the end of the semester was that they hoped 
faculty remained as empathetic and understanding to the complexity 
in their lives as they were this Spring.” The greater attention paid to 
individual students and personal factors that impacted their learning 
positioned students as experts of their own experience. 
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We learned a lot in the crisis about how we can alter assessments to mitigate 
student anxiety and stress, focus them to desirable difficulties of content rather 
than undesirable difficulties of text anxiety or how to game multiple choice exams. 
Extended time on exams and deadlines for papers also helped with stress but 
didn’t decrease performance—and many students didn’t even use the extended 
time. The knowledge that they COULD take the time helped with stress. Let’s 
carry forward these better practices. 
-NILOA Survey Respondent 


Alternative Measures 


Along with increased flexibility and newfound appreciation for and 
understanding of the student experience came the use of alternative 
measures of student learning including increased use of reflection 
and less reliance on exams. While there was a greater emphasis on 
multi-modal learning, there was a desire to stay away from proctoring, 
and instead embrace authentic forms of assessment; providing 
students with options on how best to demonstrate their learning. 
There was a marked shift away from “memorization exams to higher 
level thinking assessments” with faculty commenting on how much 
more they enjoyed the new assignments and the information gained 
about students and their learning from them. Further, honing in on 
essential learning outcomes of a course meant that faculty better 
understood the role of assessment in course design and were more 
open to allowing students to choose the format of the assignment 
they wanted to submit; whether it was creating a podcast, writing a 
paper, or conducting a survey. Student affairs assessment identified a 
movement away from larger surveys to more “point-of-contact surveys 
and leveraging existing data”. 
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I felt that faculty started to shift from surface level assessments that students could 
easily answer with help from outside sources to assessments that required more critical 
thinking skills and assessed more than just facts. Online proctoring is only a requirement 
if we are testing rote memorization, but a new focus on assessment designs that foster 
integrity and support assessment as learning does not require dependency on remote 
test proctoring services which disproportionately marginalize minority students and 
students lacking technology resources. If nothing else, the COVID-19 remote learning 
experience helped get the conversations started about improving assessment practices 
to be about learning. 
-NILOA Survey Respondent 


Addressing Inequities 


“Addressing inequities that face our students cannot go away after 
Spring” a respondent strongly declared. Addressing inequities 
involved utilizing universal design for learning, cultural concerns 
about students’ well-being who have experienced years of inequities, 
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and a stronger mission “to identify as many barriers to equity as 
possible and do our best to eliminate said barriers.” The pandemic 
exposed inequities in student technology availability and economic 
circumstances. As one respondent noted, “Just because we have access 
to technology through university contracts doesn’t mean our students 
do and just because we offer it at a ‘discount’ doesn’t mean they can 
acquire it either. Students need free Wi-Fi and technology to actively 
participate in the university community.” Others agreed stating that 
“if it is a requirement to how we will assess students, we should give 
them the technology they need.” 


66 


COVID-19 helped us understand individual student needs rather than aggregate student 
needs, which allowed us the opportunity to look at diversity, equity, and inclusion in our 
assessment activities. I hope that remains a trend and stays a part of our institution’s 
assessment culture. I am not sure we could have shifted to it without the COVID 
crisis, but we have the opportunity now to have a unified, consistent, evidence-based 
“student first” culture that strongly encourages faculty to use more effective teaching 
and really know who our students are and what they need. 


-NILOA Survey Respondent 


Planning for the Future 


The last category of responses focused on planning for the future with 
the desire to never be unprepared for this experience again. Emergency 
plans for events at the scale of the pivot to remote instruction, plans 
for moving fully online if needed in the future, widescale pedagogy 
training for all faculty, and plans to hire for teaching and learning were 
listed as possibilities. As one respondent stated, “Professors need to 
be strong in their teaching whether online or on the ground. Bad in 
person faculty were bad online teachers so what will we do about it 
in the future?” While faculty were more involved in teaching in the 
Spring 2020 semester than they had been in the past, many noted to 
the detriment of their research, students acknowledged the difference 
in faculty involvement and wanted plans to ensure a focus on teaching 
would carry over beyond the Spring semester. 


Remaining Professional Development Needs 


The last question in the survey asked all respondents, even those 
who reported no assessment-related changes being made, to indicate 
any unmet professional development needs and 355 respondents 
provided comments. Of those, 42% stated that all their professional 
development needs had been met either in the Spring or over the 
summer and some indicated that they themselves are responsible for 
providing professional development to faculty and staff. 
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Those who indicated remaining professional development needs listed 
managerial needs such as engaging and managing staff remotely, equity 
support through equity and anti-racist training on assessment and 
teaching practices, and technology issues such as video conferencing 
support and how best to facilitate online training. Specific topics of 
focus related to assessment including: 


¢ Lab assessment online 

¢ Alternative assessment examples 

* Student Affairs assessment for a remote setting 

* Qualitative assessment 

¢ How best to handle student academic conduct in an online 
setting 

* Best practices for assessing online 

* Time to review and implement everything that has been learned 


ou ty 
Open-ended responses also shared that professional development \ 
funds were cut, and that they miss opportunities to network and 
see colleagues. There was a general desire for collaboration with 42% » 
others to share and learn together—both internal and external to the _gtated that all of their 
institution. As one respondent wrote, “I need help connecting with _ professional development [i 


our faculty development office and IT: I have no idea why Ihave been _ needs had been met in 

left out of the loop or on the side. Do I need to become a certified Spring or over the summer. a 
instructional designer when I already have the skills—don’t they see 
it is all assessment!?” Others shared their worries, concerns, loss, and 
fears. I appreciate and see you and thank you for your honesty. ~ - * 
Looking forward, future professional development needs included 

examples of how programs told the story of decisions made and 

lessons learned from Spring and Fall, along with resources and support 

to “explicitly move assessment away from standardized, centralized, 

uniform reporting, and toward contextualized, responsive processes.” 

Further, the urgency of addressing systemic barriers for students was 

raised in various ways, seeking guidance on how to use assessment data 

to “spur more challenging discussions about the systemic practices that 

we perpetuate that impact students' access to and ability to succeed in 

higher education.” 


Part II: Intersections with other Survey Reports 
and Findings 


While seemingly everyone made changes, the magnitude of the shift 
and the focus of the changes differed. In a survey of 308 chief online 
officers in May 2020 undertaken by Quality Matters, “the typical 
institution had a week or two to convert over 500 courses to remote 
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instruction, and that number rose to more than 900 at the typical 
regional public university and over 2,000 at enterprise institutions 
and research universities. Accounting for multiple sections of courses, 
which, in many cases, had their own instructors, syllabi, readings and 
activities, nearly doubles the number of courses to be converted” 
(Garrett et al, 2020, p. 8). Further, chief online officers, a position 
that one might think would exist at an institution more prepared for 
online instruction at scale, reported the pivot was challenging due to 
75% of the faculty, along with 62% of students, having low levels of 
preparedness to engage online. 50% of the faculty had never taught 
online and instructional design staff was insufficient in size to handle the 
task, leaving 61% of faculty to take the lead in remote course redesign 
(Garrett et al, 2020). Those findings are similar to other surveys that 
reported 43% of students had not taken any online classes prior to the 
Spring semester with 21% having only taken one class prior (Means et al, 
2020). Resources supported technology, professional development for 
faculty, and laptops for students, with less resources directed towards 
Internet access and accessibility accommodations for students, and 
little to no mention of learning or assessment as connected to the shift 
to remote instruction. Thus, the feeling reported by NILOA survey 
respondents of assessment being side-lined or not seen as useful to the 
pivot rings true elsewhere. 


Further, while assessment professionals reported feeling overwhelmed 
and overworked in the NILOA survey, they were not alone. A member 
survey by the Association for Institutional Research (AIR) (2020) 
indicated overwhelming surges in the number of data requests to IR, 
including a deeper dive into disaggregated data and increased requests 
to survey students, faculty, and staff. Of note, the data collection 
and analysis process were internal looking to better understand 
“our students and our faculty”, not to look externally to how peers 
were responding or to benchmark policies, a shift which aligns with 
institutional leadership that reported that only 5% were engaged in 
peer benchmarking in the pivot (Inside Higher Ed, 2020). 


Changes Made 


Changes identified by respondents to the NILOA survey mirror other 
reports, although to differing degrees of change. For instance, 60% of 
students in a May-June survey reported being given the option to take 
their classes as pass/fail (Means et al, 2020), as opposed to the 36% of 
NILOA survey respondents who reported a shift to pass/fail. Further, 
the Watermark (2020) survey found that in response to COVID-19, 
63% of respondents considered or implemented changes to grading 
policies and 40% of institutions “have or are considering reevaluation 
and adaptation to policies for transfer-in of external credits” (p. 33), 
which is in alignment with the reported flexibility shared within the 
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NILOA survey responses. Further, students reported extensions 
of assignment deadlines and flexibility in submission timelines 
(Blankstein, Frederick, & Wolff-Eisenberg, 2020), which were the top 
two changes identified in the NILOA survey. 


Much like the clarifying comments in the open-ended responses to 
modifications in course evaluations in the NILOA survey, 64% of 
respondents to the Watermark (2020) survey plan to use the results 
of course evaluations to inform changes to teaching and learning. 
The intention to use information from the Spring to inform future O nl 

developments is echoed in the survey of university presidents in March, y 3 

April, and June by Inside Higher Ed (2020), with 41% incorporating respondents— 
feedback from students and faculty to improve since Spring. 


less than 
1%— stopped 


The NILOA survey found that out of all respondents, only 3 (lessthan saggsessment 
1%) indicated that assessment had stopped altogether. However, an 


ongoing concern raised by those in the field is whether assessment altogether at 

will remain a priority. In the survey of 858 respondents from 706 ee ° ° 
different ae conducted in ve 2020 by Watermark, they their institution. 
found increased prioritization of assessing student learning that was 
expected to continue for Fall 2020. Supporting this finding, Rice 
(2020) reported a 49% increase in importance of assessment in Spring 
2020, but 61% reported no change in resources either. 


Assessment as a Priority? 


As reported in the NILOA survey, assignments were modified, and 
faculty offered additional flexibility to students. Watermark (2020) 
respondents indicated increased use of direct measures of assessment 
with 97% relying on student papers and assignments and 95% relying 
on rubrics. In support, 64% of students reported that they experienced 
frequent quizzes and assignments and flexible deadlines, but struggled 
with remote labs (Means et al, 2020) and completing field experiences 
like clinicals and teaching placements (Watermark, 2020). Students 
reported that collaborative (e., group projects), technical (such as 
labs), and specialized assignments (that required access to the library 
for instance) were the most difficult to complete, along with those that 
involved multi-media (such as creation of a video, podcast, or website) 
(Blankstein, Frederick, & Wolff-Eisenberg, 2020). 


Means and colleagues (2020) found that students were more satisfied 
with their remote course when given assignments that asked them to 
express what they had learned and what they still needed to learn, and 
were given frequent quizzes to break learning into shorter chunks. The 
more faculty utilized these approaches, the more satisfied students 
were with the course at 68% satisfied versus 50% satisfied in courses 
without them. Of the eight recommended online teaching practices 
offered at the end of their survey report, the two with the most impact 
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for students were inclusion of personal messages to students about 
how they were doing in the course and course assignments that asked 
students to reflect on what they had learned and what they still needed 
to learn (Means et al, 2020), approaches that align with meaningful, 
authentic assessment practice. 


Student Feedback and Inequities 


In the open-ended responses to the NILOA survey, respondents 
indicated a desire to continue listening to and incorporating the student 
voice in institutional decision-making. Watermark (2020) also reported 
a heightened valuing of student feedback in order to understand the 


current student experience, with 87% of participating institutions Widening 
placing a high priority on using student feedback to understand and eg 
improve the student experience. Inequities 


When students were queried about their experience, they reported 
difficulties in balancing school, work, and home responsibilities as 
found by Blankstein, Frederick, and Wolf-Eisenberg (2020) in a survey 
of 21 colleges and universities and 15,000 students. Further, students 


reported that it has hard to find a quiet space to study, with over 20% ae . 
listing reliable access to Internet as a problem. Connectivity issues was eee 
graduation 


also reported in a survey of more than 1,000 college students whose 
coursework moved from in-person to remote in the Spring, with data 
collected between May and June (Means et al, 2020). Findings stated a aan 

that while 95% of students had Internet, 44% experienced connectivity earnlessat 

issues that were serious enough to interfere with their ability to attend or age 35 

participate in their courses and 23% experienced hardware or software 

issues. While 55% of students reported having a hard time finding a 

place to do their course, 54% had a hard time fitting the course in with Students experienced COVID 
their home and family responsibilities, 31% had a hard time fitting it 
into their work schedule, and 45% felt too unwell mentally to participate 
(Means et al, 2020). 


29% 


differently by race, ethnicity, 
and socio-economic status 
(Aucejo et al, 2020). 


Due to compounding factors, students are questioning whether to 
return to campus and what the impacts of the pandemic mean to them 
personally, physically, and mentally. A survey of 1,500 students at a large 
public institution that examined the causal impact of the pandemic on 
students’ current and expected educational outcomes found 13% delayed 
graduation; 40% lost a job, internship, or job offer; and 29% expect to 
earn less at age 35, furthering socioeconomic divides (Aucejo et al, 2020). 
More troubling, challenges were not uniformly distributed across college 
students with minority students reporting higher levels of stress and 
need. White students were more able than their minority student peers 
to get access to computers and laptops which is concerning because, “it 
is clear that it would have been extremely difficult—if not impossible— 
to participate in higher education during the prior term without a 
computer or laptop” (Blankstein, Frederick, & Wolff-Eisenberg, 2020, 
p. 8). 
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The issues students are experiencing and grappling with today are 
not new, they have simply been exposed, exacerbated, and amplified 
(APLU, 2020). What is concerning is that in the NILOA survey for 
Spring 2020, concerns about equity, including student voices, and 
listening to those voices before making changes was something less 
than 30% of respondents strongly agreed with. Yet, 90% of university 
presidents reported being very concerned about the inequitable impact 
on minority students but only 45% felt they were successful in ensuring 
equitable access (Inside Higher Ed, 2020). 
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Due to the uneven impact COVID-19 is having on communities of color both in terms 
of infection rates and treatment, as well as in terms of accessibility and financial stress, 
we can easily foresee dramatic reductions in the number of minority and first-generation 
students who will be able to return to school next year due to family conditions. For those 
who are able to return, there will undoubtedly be a need for additional support including 
academic, financial, and emotional supports...(p. 5)...As we move forward as an industry 
into uncharted waters, the question will be not just how do we in public institutions 
of higher education respond to COVID-19, but how do we respond to the underlying, 
systemic challenges that have been long known but rarely successfully addressed? (APLU, 
2020, p. 7). 


Maslow’s Hierarchy of Needs 


Concerns about mental health, well-being, housing, and food security 
were rampant in survey reports (Blankstein, Frederick, & Wolff 
Eisenberg, 2020; APLU, 2020, Inside Higher Ed, 2020). Maslow’s 
(1943) theory of human motivation presents an order in which needs 
need to be met for functioning well, let alone learning. The theory 
begins with three foundational needs: physiological needs such as food, 
water, sleep, and clothing; then safety needs such as personal security, 
employment, and health; followed by love needs such as friendship, 
sense of connection, and family. All of the foundational needs have 
been compromised for students, and some of our faculty and staff too, 
which leads to emergency reactions (such as leaving education) and 
requires additional support. 


There are pandemic driven class, race and ethnicity inequalities in who 
lost jobs, who is more likely to get sick, and which students are also 
more likely to have a family member who lost a job, got/is sick, or 
died (Matthews, 2020). With so many compounding factors occurring 
to students at once, the Higher Education Data Sharing Consortium 
reported that students who were considering leaving an institution pre- 
COVID had those feelings amplified (Blaich & Wise, 20204, 2020b, 
2020¢). Further, they report that 21% of students worry about having 
a safe place to sleep every night—a concern about a basic human need 
that is higher for students of color (Blaich & Wise, 2020a, 2020b, 
2020C). 
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The compounding factors form a nightmare situation which a single 
“flexible” semester will not alleviate or address. In the Fall of 2019, 558 
APLU members were surveyed with follow-up phone interviews in 2020, 
68% reported concerns for student mental health and well-being, with 
63% indicated diversity and inclusion efforts were a challenge (APLU, 
2020). However, students of color have experienced a disproportionate 
impact on their mental health and well-being (Inside Higher Ed, 2020). 
In a survey of students, students who were also caregivers were doubly 
worried about physical and mental health, and students concerned 
with one basic set of needs (such as having enough food or money to 
buy food, afford rent or mortgage, or pay utility bills) were generally 
concerned about the entire set, with students who are Middle Eastern or 
North African, American Indian or Alaska native, and Black or African 
American reporting the highest levels of concern about their physical 
health (Blankstein, Frederick, & Wolff-Eisenberg, 2020, p. 13). Concerns 
that existed pre-COVID have been amplified, basic student needs are 
not met, and the rates at which they are not met are nearly double for 
students of color. 


Part III: Do’s and Dont’s for the Future 


After examining the findings from the NILOA survey of assessment- 
related changes in Spring 2020, along with the various reports that have 
been released between March and July exploring institutional, faculty, 
and student responses and experiences, I offer a list of 5 Do’s and 5 Do 
Not’s to help guide us through the uncertain Fall and onward to Spring. 


Don’ts 


1. Do not forget that we are in a pandemic. Still. Do not forget that it 
is also an inequitable pandemic. 

2. Do not cause further harm. Do not support, enable, or endorse 
policies that perpetuate further inequities or fuel negative 
perceptions of students. 

3. Do not ask students for their approval of a decision that has already 
been made. Instead, engage with them in advance to help determine 
a solution. 

4. Do not require a higher-level of proof of learning in an online class 
than you would normally require in a face-to-face setting. 

5. Do not forget that this is not the educational experience students 
wanted or expected. Nor is this a test of online education. And in 
case you were wondering, it still will not be “online education” in 
the Fall. It will continue to be a derivative of emergency remote 
teaching and learning. 


To begin, the pandemic is not over. The triage response period in the 
Spring has passed, the full ripples from which remain unknown, but 
attempting to live, survive, and learn in a pandemic is ongoing. This 


Concerns that 


existed pre- 
COVID have 


been amplified, 


basic student 
needs are not 
met, and the 
rates at which 
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are nearly double 
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means that expectations to maintain the same levels of productivity 
in research and teaching for faculty, and learning for students, is not 
only unhelpful, it is unhealthy, likely to fuel mental health issues, and 
preposterous during a pandemic. This does not mean that students 
are not learning, it means institutions cannot afford to act as though 
intentional instructional design will alleviate the concerns of family 
members getting sick, professors falling ill, mounting mental health 
issues, job and food insecurities, and other compounded disruptions 
and barriers that have been raised between students and their ability 
to have a singular focus on learning. Further, administrators must 
be mindful of the prolonged stress and fatigue of adapting to the 
shifting environment for their employees and students. Our faculty 
were sprinters and now have become marathon runners without the 
training, tools, and mental support to continue at the required pace. 


While the Spring was not about online education or online learning 
but about learning in a pandemic during a quick pivot to remote 
instruction, the Fall is not “online learning” either. Students are not 
prepared to learn online in terms of meeting proper technology and 
bandwidth requirements, and best practices in online learning and 
assessment do not apply to current inequitable situations and pandemic 
lived circumstances. As one chief online officer wrote, “‘Underprepared 
students’ doesn’t exactly capture how I’d characterize the challenge— 
more like ‘students who didn’t sign up for online courses and were 
asked to engage with them, often in far from ideal circumstances’ “ 
(Garrett et al, 2020, p. 11). Instead, limping into the Fall together will 
be tired and stressed faculty and students, still not ready for learning 
online, which makes it all the more important to not do further harm 
through policies and practices. 


As one NILOA survey respondent wrote, “Listening to student’s 
needs did not dominate our campus response to COVID. As a result, 
many students have withdrawn this quarter, expressing issues with 
technology, health services, housing, disability support services, and in- 
person connection to faculty. We did not understand their experience 
or who our students were.” Compassion and understanding of who our 
students are should drive decisions as opposed to starting from a point 
of assuming the worst of our students. Instead of designing learning 
to block perceptions of increased online cheating (Lederman, 2020) 
or assuming that because a pass/fail option is offered to students that 
they would disengage from their learning without the blunt force of a 
grade to drive them, built upon the perception that students do not 
want to or will not work hard or learn, we need to develop policies and 
practices that support and help students. 


Such negative mentality towards students misses the connected and 
contextual nature of the student lived experience that directly impacts 
their learning. For instance, while a student may not have lost a job, 


“Listening to student’s 
needs did not dominate 


our campus response 
to COVID. As a result, 
many students have 
withdrawn this quarter, 
expressing issues with 
technology, health 
services, housing, 
disability support 
services, and in-person 
connection to faculty. 
We did not understand 
their experience or who 
our students were.” 


-NILOA Survey Respondent 
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the parent who is paying for the education may have and, thus, the 
student cannot afford to attend class at a specific time due to extra 
hours picked up at work that now conflict with their education; or 
maybe the student transfers and attends somewhere closer to home 
(Mangan, 2020). If we make the choice between family and living or 
education, given Maslow’s hierarchy of needs mentioned prior, living 
will win over education. It need not be an either/or. 


We have an entire generation of students living through “crisis-driven 

policy shifts” (Basken, 2020). Policies that punish students for making 

difficult choices, such as deciding to utilize a pass/fail option during a 

time of crisis, should not be carried forward. Concerns have already 

been raised as to whether institutions of higher education will accept 

pass/fail credit during the forthcoming “Corona swirl” (Neutuch, 

2020; Reed, 2020; Fink, 2020). Matt Reed (2020) argues that while a 

single semester with an asterisk on it is not likely to do lasting harm, ° 

one that lingers may. The lingering means that there may be students The choice for 
who graduate with the majority of their time spent learning during a students should 
pandemic, bringing into question the signaling function of GPA (Reed, : 

2020). In the Spring semester, more than 150 institutions adopted a not be either 
pass/fail grading system (Basken, 2020) and while assessment of student 

learning can provide relief to the noise added by pass/fail signaling in health and safety 
our traditional GPA-based system, the more important option is to do or school, work 
no further harm by adding additional barriers to student movement, 


options, and hard choices. or education, or 
ae family or college. 


| | | It need not be an 

1. Do use learning outcomes as a guide and means to design and . 

focus educational offerings. either/or. 
2. Do listen to student voices AND respond accordingly. 
3. Do modify assignments and assessments in ways that are flexible, 

utilize low-bandwidth, and are based in the principles of equitable 

assessment. 
4. Do be aware of and address systemic inequities. 
5. Do engage in trauma-informed and healing-centered pedagogy 

and assessment. 


There is much that can be done. Even in the prior section focused 
on what not to do, there are things to do zmstead. We can continue to 
work collaboratively, sharing practices and resources with each other, 
and lifting each other up. While intentional educational design will not 
solve all problems, a clear focus on learning outcomes helps students 
focus their attention on what is most important. The Watermark (2020) 
survey reports, “Institutions should ensure that learning outcomes are 
clearly defined, that courses and curricula are designed around them, 
and that students have clear visibility into outcomes so they know 
what is expected of them and what they can expect from the course” 
(p. 13). That transparency will matter more to students now than ever. 
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We can use curriculum maps in programmatic planning and advising, 
remain flexible, and modify offerings based on learning outcomes. 
In some ways, it is a return to assessment basics—being clear about 
what is most important for students to learn in the context of what is 
reasonable and doable for students and faculty. 


Student voice can take on the mantle of feed forward as opposed to 
feedback on what is needed and what would be feasible during this 
time. Including student voice in the decision-making process, looking 
internally to see who our students are and what they need, will ensure 
connected and supportive learning environments. We should never 
again be surprised about our students’ circumstances because we did 
not know who they were beyond aggregate numbers. Further, while 
student voices are used to feed forward our next steps, all the data 
collection and survey findings shared in this report were done via 
online or electronic surveys. If our only means of connecting with or 
gathering data from students is virtual or online, not by other means 
such as phone or mail, we do not hear from students without access 
to Internet or who have limited capacity to respond. They become 
invisible. To avoid missing important student voices, multiple data 
collection mechanisms should be employed, which will also help to 
avoid survey fatigue on the part of students. 


Flexibility in assignments and assessments can serve to help support 
student learning. Coupled with trauma-informed and healing-centered 
pedagogy and assessment, faculty and staff can partner with students 
as producers of content, experts of their lived experience, and be 
active partners in solving the problem of demonstrating complex 
learning during a pandemic. Such adaptability began in the Spring and 
will likely continue forward, using low-bandwidth assignments that 
encourage co-creation and peer review. Students built and improved 
wiki pages on key topics. Instead of completing an archival research 
project, they gathered materials to submit to the archive about their 
COVID-19 experience. They can examine their Internet speed and 
engage quantitative reasoning skills and oral communication while 
advocating for themselves to customer service (Taylor, 2014), they can 
gather oral histories of their family, document through photography 
the way they see the world, engage in reflection, and even disaster 
analysis (Penn, 2015). 


Lastly, we can address systemic inequities, beginning with simply 
accepting transfer credit regardless of what it is. With students likely 
to stay closer to home, shift institutions, utilize pass/fail or credit/ 
no credit options (Fink et al, 2020) the sea of red tape before them 
is already boiling. In an April 16, 2020 statement by six Washington, 
D.C.-based higher education organizations, principles on acceptance 
of credit were shared indicating in part: 


“Many faculty at my 
institution are reporting 
unsubstantiated claims in 


increases of cheating due 
to the remote conditions 
of taking tests and such. 
That can lead to bad policy 
decisions, but worse it 
divides our faculty and 
students further from each 
other.” 


-~NILOA Survey Respondent 
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1. Institutional policies and the evaluation of grades and credit 
should recognize the extraordinary burden placed on students 
during this time. Even in the best of cases, student dislocation 
and the need to change the very basic patterns of life impose 
challenges on our students that may have an impact on their 
performance. 

2. Institutional policies and practices should recognize that 
traditional inequities are exacerbated in the current crisis and 
that “equal” treatment of students’ transcripts is unlikely to 
result in “equitable” outcomes. 

3. Institutional policies and practices should, therefore, be as 
holistic as possible, taking into account the range of situational 
and behavioral circumstances in which our students find 
themselves. 

~ From Statement on Principles of Acceptance of Credit 


Final Thoughts 


Higher education is faced with a series of choices. The choice of whether 
to embed learning outcomes as an integral element of educational 
design, the assessment of which then provides information to faculty, 
staff, and students for learning, planning, and future directions. The 
choice of whether to address persistent systemic and systematic 
inequities—which existed before but have been clearly exposed. The 
choice of whether to transform; choosing systems over silos and doing 
right by students instead of blaming students for not intuiting how 
to navigate rules and structures that were never built to ensure their 
success. 


Fall 2020 will be difficult and uncertain. The flexibility demonstrated 
in the Spring will likely carry over, and the field will continue to share 
with and learn from one another. May we act with grace and patience, 
with ourselves and our students, and continue to learn how to truly 


fulfill our institutional missions of teaching, learning and service. 
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Appendix A. Survey Methodology and Sample 


In May-June 2020, NILOA released a brief electronic questionnaire on COVID-19 assessment-related 
changes made in Spring 2020. The questionnaire was announced via the NILOA newsletter and shared on 
various assessment related listservs and through partner newsletters. A total of 834 respondents completed 
the survey. Upon cleaning the data, a total of 813 responses were retained for analysis. The 813 total responses 
included 624 different institutions and organizations with representation from all 50 US. states and two 
territories, as well as 18 international respondents from Algeria, Bahrain, Canada, Ecuador, India, Italy, 
Jamaica, Japan, Mexico, Nigeria, Saudi Arabia, South Africa, and the United Kingdom. 


The survey respondents were those with assessment as their primary job responsibilities (Table A). Several 
categories were collapsed into those in the table below, based on grouping together job functions and 
responses that were similar. As such, Administrators includes staff, except for librarians who were closer 
in response to faculty and whose responses are part of the faculty grouping. Assessment professionals also 
include Institutional Researchers with oversight of assessment and student affairs specific assessment 
professionals. There were 4 students who completed the survey as well, however, due to the small nature of 
the sample they are not included in further analyses for statistical significance in different responses, but 
differences in their responses from faculty, administrators, and assessment professionals are noted in the 
report narrative based on descriptive statistics. 


Primary Assessment-Focused Role 
Assessment Professionals (Academic and/or Student Affairs) 


Faculty 


Student 1% 


Did not Identify 


Table A. Demographics of respondents. 


Data were analyzed for descriptive statistics as well as for statistical significance on a variety of key variables. 
Of note, differences by regional accreditation were not examined as all regional accreditors (prior to the July 
I name change to accreditors) were flexible in allowing institutional responses and the pandemic was global 
in nature. As such, it is unlikely that in this survey, accreditation region would be a variable of distinctive in- 
terest. Instead, variables found to be impactful in other COVID-19 survey reports were examined including 
selectivity, urban/rural location, control, residential/non-residential, and institutional type in order to run 
the additional analyses. An option was provided for survey respondents to provide their institution name, 
which was shared by 574. For those respondents who provided their institution name (n=574), the institution 
IPEDS number was matched and additional information was attached to the survey responses. 


Location 


* 79% were urban (meaning cities or suburbs) 
¢ 21% were rural (meaning towns or rural locales) 
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Selectivity 


* 53% were inclusive in admissions 
° 47% were selective in admissions 


Residency 


° 51% were residential campuses 
* 49% were non-residential or commuter campuses 


Highest Degree-Granted 


* 24% Associate degree-granting institutions 

° 15% Baccalaureate degree-granting institutions 
* 23% Master’s degree-granting institutions 

* 32% Doctoral degree-granting institutions 

° 6% Specialized degree-granting institutions 


Control 
° 66% were public 
° 32% were private 
° 2% were for-profit 
Of note, 7% of respondents were from land-grant institutions, 13% from medical institutions, 3% from 


Historically Black Colleges and Universities, 8% from Hispanic-Serving Institutions, 14% overall from 
Minority-Serving Institutions, 1% from women’s colleges, and there was 1 Tribal College respondent. 
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Appendix B. Questionnaire 


NILOA COVID-19 Survey of Assessment-Related Changes Made in Spring 2020 


Please read and click "Yes" below if you are willing to participate. 


Purpose: As a result of COVID-19, the higher education landscape drastically shifted, including 
assessment related processes, practices, reporting mechanisms, and assessment of student learning 
itself. This survey, administered by the National Institute for Learning Outcomes Assessment (NILOA), 
is designed to capture a snapshot of assessment-related changes made during Spring 2020 in 
response to the sudden shift to remote instruction and to inform future professional development 
needs. 


Confidentiality and Voluntary Nature: Your participation in this research is completely voluntary. The 
questionnaire will take approximately 3-5 minutes to complete. The study involves no foreseeable 
risks and you may exit at any point in time. Your responses will be confidential. This survey has been 
approved by the UIUC Institutional Review Board (IRB). If you have any questions or comments about 
the survey, please contact Dr. Natasha Jankowski at the National Institute for Learning Outcomes 
Assessment: niloa@education. illinois.edu or (217) 244-2155. 


* Do you agree to participate in this survey? 


) Yes 


) No 


NILOA COVID-19 Survey of Assessment-Related Changes Made in Spring 2020 


Welcome! Please note, due to the short length of this questionnaire, once you advance to the next 
page you will not be able to return to the prior page and/or modify previously submitted answers. 
Thank you for your participation and all that you do to support student learning. 


* What is your primary role? 
) Administrator 

Assessment Professional 

Faculty 


Student Affairs 


\ 


Student 


) Staff 


Other (please specify) 
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\ 


* Institution Name. No institutional responses will be shared. Institution name is requested for NILOA 
researchers to connect responses with IPEDS data, based on institutional name, for variables on size, control, 
and type to disaggregate survey responses for analysis. 


* What changes were made to assessment related processes and practices for Spring 2020? Select all that 
apply. 


None 


Modification of assessment reporting processes or reporting questions 


Changes to timing of submission of assessment reports 


Modification of questions on course evaluations 


Shift to pass/fail for Spring term/semester 


Shift to credit/no credit for Spring term/semester 


Changes to your assessment-related roles and responsibilities 


Modifications to assignments or assessments 


Acceptance of alternative assignments for Spring courses 


Flexibility in submission deadlines for assignments/assessments 


Other (please specify) 


NILOA COVID-19 Survey of Assessment-Related Changes Made in Spring 2020 
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The following statements address possible drivers behind the assessment-related changes made in the prior 
question. Due to NILOA’s focus in the past several years on equity and assessment, as well as student 
involvement in assessment, the statements focus specifically on the possible decision drivers of equity and 
student needs. For each statement, indicate degree of agreement from Strongly Agree, Agree, Disagree, 
Strongly Disagree or N/A. 


Strongly Agree Agree Disagree Strongly Disagree N/A 


Assessment-related 
changes were 
undertaken to address 
student needs. 


Equity concerns drove 
the decisions to make 
changes. 


Concerns about 
students’ differential 
access to technology 
(including reliable 
Internet) were 
determining factors in 
making changes. 


Concerns about 

students’ ability to learn 

in their remote i Fant ic 
environments drove 4 
decisions to make 

changes. 


Students were invited to 
identify needs (via 
survey, focus group, 
phone calls, etc.) prior to 
decisions being made. 


Information gathered 

from students influenced lan a van AF ran 
decisions on what to — 
change. 


NILOA COVID-19 Survey of Assessment-Related Changes Made in Spring 2020 


Are you concerned the changes made in response to COVID-19 will negatively impact the culture of 
assessment at your institution? 


> Yes 


~) No 
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NILOA COVID-19 Survey of Assessment-Related Changes Made in Spring 2020 


If yes, in what ways? 


NILOA COVID-19 Survey of Assessment-Related Changes Made in Spring 2020 


What changes, if any, would you like to see continue past the Spring semester to support student learning 
during COVID-19? 


NILOA COVID-19 Survey of Assessment-Related Changes Made in Spring 2020 


What professional development needs have not yet been met for you? 
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